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ABSTRACT* 

Objectives: This pilot study was designed to 
evaluate the impact of a pre-rotation workshop 
(PRW) on pharmacy students' clinical skills and 
preparation for clinical Advanced Pharmacy 
Practice Experiences (APPE) involving direct 
patient care. 

Methods: Randomized controlled trial of an 
educational intervention with Institutional Review 
Board approval. PRW activities designed to 
simulate rotation activities around five 
competencies, patient charts, medication histories, 
SOAP notes, patient presentations, and 
professionalism. Endpoints were evaluated using 
clinical rotation preceptors' evaluation of 
performance and students' performance on 
objective structured clinical exams (OSCE). 
Results: Eight fourth-year students and eight CPA 
matched controls (20% of the total class) were 
selected to voluntarily participate. The PRW 
demonstrated a positive impact on students' clinical 
skills and preparation for rotations by improving 
OSCE performance. However, no significant 
differences were found between groups when 
comparing preceptor evaluations of skills on 
rotations. These results are limited by the small 
sample size, potential OSCE "test-wiseness" 
effects, lack of OSCE evaluator blinding to study 
groups, potential case specificity effects due to the 
limited number of cases used on the OSCE and 
possible lack of sensitivity of the rotation evaluation 
tool to capture true differences among the 
experimental and control group participants. 
Conclusion: The PRW was successful at advancing 
students' clinical skills and preparation for rotations 
and may be considered as a tool to help bridge 
didactic to clinical experiences in the Pharm.D. 
curriculum. 
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EVALUACION DEL IMPACTO DE UN 
TALLER PRE-ROTACION EN LA 
PREPARACION DE LOS ESTUDIANTES 
PARA LAS PRACTICAS CLINICAS 
AVANZADAS DE FARMACL4 PRACTICA 

RESUMEN 

Objetivos: Este estudio piloto se diseno para 
evaluar el impacto de un taller pre-rotacion (TPR) 
en las habilidades clmicas de los estudiantes de 
farmacia y en su preparacion para las Practicas 
Clinicas Avanzadas de Farmacia (PAF) que 
envuelven atencion directa al paciente. 
Metodos: Ensayo aleatorizado controlado de una 
intervencion educativa con aprobacion de la Junta 
de Revision Institucional. Las actividades del TPR 
se disenaron para simular las actividades de la 
rotacion sobre 5 competencias: historias clinicas, 
historiales farmacoterapeuticos, notas SOAP, 
presentaciones de pacientes, y profesionalismo. Los 
resultados se evaluaron utilizando la evaluacion de 
tutores de practicas clinicas y examenes clinicos 
estructurados objetivos (OSCE). 
Resultados: Se seleccionaron 8 estudiantes de 
cuarto aiio y 8 GPA controles emparejados (20% 
del total e la clase) para participar voluntariamente. 
El TPR demostro un impacto positivo en las 
habilidades clinicas de los estudiantes y en la 
preparacion para sus rotaciones mejorando el 
desempeno en el OSCE. Sin embargo, no se 
encontraron diferencias significativas entre los 
grupos cuando se compararon las evaluaciones de 
los tutores de las habilidades en las rotaciones. 
Estos resultados estan limitados por el pequeno 
tamano de la muestra, posibles efectos de 
conocimiento de los examenes OSCE, la falta de un 
evaluador OSCE ciego para los grupos de estudio, 
posibles especificidades de los casos debido al 
numero limitado de casos usados en el OSCE, y 
posible falta de sensibilidad del instrumento de 
evaluacion de la rotacion para encontrar verdaderas 
diferencias entre los grupos control e intervencion. 
Conclusion: El TPR tuvo exito en aumentar las 
habilidades clinicas de los estudiantes y la 
preparacion para la rotaciones, y podria 
considerarse una herramienta para reducir la brecha 
didactica a la practica clinica en el curriculo de 
Doctor en Farmacia. 
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patient care activities while on the clinical rotations. 
The results of this study should indicate if a PRW 
prepares students for the requirements of a clinical 
APPE and if it could be considered as an option to 
help bridge didactic to clinical experiences. 

The study objective was to evaluate the impact of a 
PRW on four elements: students' clinical skills; 
students' preparation for clinical APPEs; student 
performance on clinical APPEs; and students' 
retention of clinical skills. 

METHODS 
Participants 

The University of Oklahoma (OU) College of 
Pharmacy is a 4 year (P1-P4) doctor of pharmacy 
program that has two campuses with approximately 
80 students at the Oklahoma City campus (OKC) 
and 60 students at the Tulsa campus. At the time of 
this study, no fourth year students were enrolled on 
the Tulsa campus, therefore only OKC fourth year 
students were available. Students participate in 
introductory pharmacy practice experiences (IPPE) 
during the first through 3 years of the curriculum. 
During the fourth year, students complete 9 months 
of advanced pharmacy practice experience (APPE) 
which consist of spending one month at nine 
different sites. 

The study was aimed at fourth year (P4) students 
preparing to enter APPE's and was designed as a 
randomized controlled trial using the PRW as an 
experimental educational intervention. The study 
was approved by the University of Oklahoma 
Institutional Review Board, and informed consent 
was obtained from all study participants. All 
students in the fourth year of the entry-level Doctor 
of Pharmacy professional degree program (n=82) 
were eligible for the study. Ten percent of the class 
(N=8) was randomly selected using a table of 
random numbers and only those randomly selected 
students were invited to voluntarily participate in the 
experimental group of the study. The CPAs of the 
randomly selected students were then reviewed and 
categorized into low (2.0-2.5), moderate (2.5-3.0) 
and high (3.0 to 3.5) GPAs. The entire class was 
then organized according to these GPA guidelines 
and ten percent of the class (N=8) were randomly 
selected to voluntarily participate in the study to 
serve as a GPA matched control for the 
experimental group. Student participation was 
anonymous for all students and faculty outside of 
the experimental group and study investigators. 

Intervention 

The pre-rotation workshop was developed and 
administered by five clinical faculty members and 
one instructional design and assessment faculty 
member. The workshop was delivered as three 
days of eight-hour sessions led by the study 
investigators with a student to faculty ratio of three 
to one. The workshop topics were developed 
around five core areas that were determined to be 
assumed competencies at the start of clinical 
rotations: patient charts, medication histories, 
subjective objective assessment plan (SOAP) 



INTRODUCTION 

The Advanced Pharmacy Practice Experiences 
(APPE) comprise at least twenty-five percent of the 
Doctor of Pharmacy (Pharm.D.) curriculum and can 
be considered the capstone component that 
assures students have achieved the professional 
competencies necessary to practice pharmacy.^ 
Furthermore, it is stated that these experiences 
must integrate, apply, reinforce, and advance the 
knowledge, skills, attitudes, and values developed 
through the other components of the curriculum. 
These experiences are required to provide direct 
interaction with patients and collaboration with other 
health care professionals and to include 
experiences in both acute care general medicine 
and ambulatory care settings. These clinical APPEs 
are designed to allow students to refine their 
professional competencies by allowing them to 
broaden their experience in patient care 
responsibilities, including conducting patient 
interviews and education, communicating and 
documenting recommendations for medication 
therapy, and participating in interdisciplinary team 
rounds. As such, colleges must teach clinical skills 
throughout the curriculum in order to bridge the 
traditional didactic and clinical portions and to 
ensure students enter the APPE year with baseline 
knowledge, skills, attitudes, and values required to 
perform the final year of patient care 
responsibilities. Students without those baseline 
professional competencies would not only perform 
very poorly on these rotations but also interrupt the 
delivery of patient care in these patient care 
settings. 

Preparedness for the APPEs was identified to be a 
concern at our institution among clinical faculty 
preceptors through professional development 
discussions. Some described as much as one week 
of a four-week rotation was spent reviewing the 
expectations and skills needed to successfully 
complete the rotation, and all believed this absorbed 
valuable time away from applying those skills in the 
patient care setting.^ These discussions delineated 
five areas of professional competencies of which 
students were expected to enter rotations with 
baseline knowledge: patient charts, medication 
histories, SOAP notes, patient presentations, and 
professionalism. Several methods could be 
considered to interface clinical skills with the 
didactic coursework, including the structure and 
delivery of the Introductory Pharmacy Practice 
Experiences (IPPE), the utilization of portfolios, 
clinical coursework, pre-rotation training, as well as 
many other educational tools. 

The current pilot study was designed to evaluate the 
impact of a pre-rotation workshop (PRW) on fourth 
year students' ability to demonstrate preparedness 
for clinical APPEs and to better apply those skills to 
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notes, patient presentations, and professionalism. 
Tlie worl<sliop activities were designed to simulate 
rotation activities with brief discussions with faculty 
(all topics), role-play scenarios in patient care 
activities (medication histories), and self-directed 
learning assignments (patient charts, SOAP notes, 
patient presentations). The simulated patient charts 
contained samples of a fictitious patient's health 
related material (history and physical report, 
progress notes, physician's orders, medication 
administration records, laboratory results, vital 
signs, and microbiology reports). All activities were 
assessed by the study investigators using 
standardized rubrics created and tested internally, 
and participants were given verbal and written 
feedback on all topics and assignments. 

Endpoints 

There were two primary endpoints in this study: the 
impact of the PRW on students' clinical skills and on 
students' preparation for rotations. 

The impact of the PRW on students' clinical skills 
was determined by comparing the experimental 
group's performance on an objective structured 
clinical exam (OSCE) prior to and following the 
PRW. The OSCE was a standardized scenario 
reflecting a commonly encountered problem on a 
clinical rotation. This scenario focused more on 
professional skills emphasized in the workshop than 
on specialized knowledge and was chosen as the 
assessment tool due to its simulation of a clinical 
rotation interventions and integration of all clinical 
skills included in the PRW. Two OSCE cases 
were used in this study: case 1 was used in the 
experimental group prior to PRW and case 2 was 
used in the experimental and control groups prior to 
and following the year of APPE rotations. The use 
of two different OSCE cases (cases 1 and 2) to 
evaluate the impact of the PRW on the experimental 
group's clinical skills (a primary endpoint) was to 
avoid the possibility of immediate recall of case 
specifics over the three days of PRW. 

The other primary endpoint, impact of the PRW on 
students' preparation for rotations, was determined 
by comparing the experimental and control groups 
using two endpoints: First, performance on an 
OSCE (case 2) at the start of the APPE rotations 
year was compared between the study and control 
groups. Second, the two groups were compared 
using a standardized evaluation of students' rotation 
performance completed by faculty preceptors during 
the first week of a clinical APPE (involving direct 
patient care responsibilities). Every attempt was 
made to blind faculty preceptors to PRW 
participation. A small portion of faculty preceptors (3 
out of 22 total) included study investigators; 
therefore it was impossible to blind every preceptor. 
However, student assignments to clinical rotations 
were completely random. All participants were 
evaluated using a standardized form, which 
assessed the five core areas of competency (patient 
charts, medication histories, SOAP notes, patient 
presentations, and professionalism) with a rating 
scale of 1 (low) to 7 (high). 



Secondary endpoints included the impact of the 
PRW on APPE performance and on retention of 
clinical skills. Impact of the PRW on rotation 
performance was evaluated by comparing 
experimental to control groups using the previously 
described standardized evaluation of students' 
rotation performance completed by faculty 
preceptors upon completion of a clinical APPE. 
Retention of clinical skills was evaluated by 
comparing the experimental and control groups' 
performance on an OSCE at the end of fourth year 
APPE rotations. For this endpoint, the second 
OSCE case (case 2) was administered at both the 
start and completion of the year of APPE rotations 
in both groups. The possibility of students recalling 
specifics of case 2 as a confounding variable was 
not considered to be an issue because of the time 
that elapsed for this endpoint. The use of the same 
case was thought to help minimize differences in 
student specific knowledge as well. 

Data Analysis 

Performance on the OSCE was graded by the study 
investigators using a grading rubric specific to the 
two cases (cases 1 and 2) used in the study. 
Results were then interpreted to be passing (grade 
S70%) or failing (grade <70%). The pass rate of the 
PRW group was compared using the paired 
samples t-test. When the PRW group pass rate was 
compared to the control group, an independent 
samples t-test was used. 

Rotation performance was evaluated using the 
standardized evaluation form completed by rotation 
preceptors. The responses were evaluated by the 
study investigators after completion of study follow- 
up and were categorized as having an either 
acceptable or unacceptable level of skills for 
rotations. An acceptable level of skill was defined as 
an average score of five to seven (overall and within 
each of the five categories), and an unacceptable 
level was defined as an average score of one to 
four. Percentages of students in each category were 
compared using the independent samples t test. 

RESULTS 

A total of sixteen students were recruited from the 
fourth year of the entry-level Doctor of Pharmacy 
professional degree program: Eight students (13% 
of the class) in the experimental group and eight 
students in the control group. Experimental and 
control group participants were categorized into the 
low GPA (Experimental=N=1 and control=N=1), 
moderate GPA (N=3 and 3), and the high GPA (N=4 
and 4). All participants in the experimental group 
completed the PRW, including the OSCE cases 
both before (case 1 ) and after the PRW (case 2). All 
participants in both the experimental and control 
groups were administered the OSCE case (case 2) 
both before and after the year of APPE rotations. 
Twenty-two full-time college of pharmacy faculty 
completed a total of 47 preceptor evaluations of 
rotation preparation (completed at the beginning 
and end of the rotation). All participants were 
evaluated on at least two clinical APPEs (range 2-5 
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evaluations conducted per participant, average 
number of clinical rotations evaluated =2.94). 

The difference in the experimental group's 
performance on an OSCE prior to (38% pass rate) 
and following (88% pass rate) the PRW was 
statistically significant (p<0.05) measured by the 
paired samples t-test. In addition, there was a 
statistically significant difference between the 
experimental group's performance on an OSCE 
prior to start of the APPEs (88% pass rate) 
compared to the control group's performance (0% 
pass rate) (p=0.001) measured by the independent 



samples t-test. However, there was no statistically 
significant difference in performance on an OSCE 
after the completion of the APPEs between the 
experimental (75% pass rate) and control (50% 
pass rate) groups (p=0.252) measured by the 
independent samples t-test. No statistically 
significant difference was found between the 
experimental and control group's demonstration of 
clinical skills during the first week or at the 
completion of the clinical APPEs (see table 1) as 
evaluated by the rotation preceptors. 



Table 1 : Clinical Rotations Evaluations. 




PRW Group (N=8) 


Control Group (N=8) 




Mean^ (SD) 


Mean^ (SD) 


Beginning of Rotation Month Evaluation Results 


Average - 1^' clinical rotation 


3.61 (0.94)* 


4.10(1.10)* 


Average - All clinical rotations 


4.38 (1.14)* 


4.47(1.10)* 


End of Rotation Month Evaluation Results 


Average clinical rotation 


5.54 (0.78)* 


5.06 (0.90)* 


Average - All clinical rotations 


5.73 (0.69)* 


5.54 (0.88)* 


t Based on a 7-point Likert scale ranging from 1 = low to 7 = high 




*no statistically significant differences between PRW and Control groups 





DISCUSSION 

The PRW was designed to be an introductory APPE 
for the transition from didactic coursework to clinical 
APPEs at our institution. The statistically significant 
difference in pre and post PRW OSCE results 
reveal that the PRW was demonstrated to be an 
effective method to improve students' clinical skills, 
evidenced by the improvement in OSCE 
performance in the experimental group in this study. 
This finding would be expected because the 
workshop provided participants with intense, 
prolonged practice with direct feedback from clinical 
faculty. 

The first important finding from this study is the 
impact of the PRW on students' preparedness for 
clinical APPEs, as this was the underlying 
motivating issue at our institution. The results from 
this study suggest that the majority of students in 
this study were not prepared for clinical APPEs, 
evidenced by the student's low pass rates (38% of 
the experimental group and 0% of the control group) 
on their first OSCE in this study taken prior to the 
start of the APPE year. The second important 
finding is that the study results suggest that the 
PRW as a three-day introduction rotation was 
effective at preparing the experimental group of 
students for the clinical skill requirements for the 
APPEs, evidenced by 88% of PRW students 
passing the OSCE. 

While this result indicates the need for more 
experiences to help bridge didactic to clinical 
experiences throughout the curriculum, the result is 
limited by the small experimental group sample 
size. The use of only two OSCE cases also limits 
these study results. Although the two cases 
addressed general acute care issues and clinical 
skills with a similar level of complexity that all fourth 
year students must possess for APPE and should 



not have afforded an advantage to the experimental 
group, OSCEs were not commonly used as an 
evaluation tool with this class of students during the 
time of the study in any phase of the curriculum. 
Since none of the study participants had experience 
with OSCE exams prior to the study, the 
experimental group's experience in taking the pre- 
workshop exam might (by itself) have given that 
group an advantage ("test-wiseness") over the 
control group when both groups took the case 2 
OSCE exam since the experimental group may 
have had less apprehension and more familiarity 
with exam procedures as described by Mahamed, 
Gregory and Austin previously.^" 

The third limit to this finding is that the OSCE 
examiners were not blind with respect to study 
group and the OSCE examiners consisted of the 
faculty who provided the pre-rotation workshop. 
Furthermore, the OSCE's were graded live by the 
evaluators instead of videotaping, therefore limiting 
scoring by other examiners. Although the examiners 
were trained with the rubric during sample OSCEs 
and demonstrated similarity in scoring, no formal 
assessment of inter-rater reliability was performed 
and the exam process was not validated. 

The lack of a difference on the preceptor's 
evaluation of student performance during the first 
week and at the rotation completion on clinical 
APPEs was discouraging; however this finding may 
point out several important limitations of this study. 
First, the use of a small, randomly selected sample 
size introduced a wide variability in assigned clinical 
rotations available to solicit these evaluations. This 
meant that students were not all entering clinical 
APPEs at the same time in the year (first clinical 
rotation ranged from June to April of that academic 
year) and that multiple faculty preceptors were 
completing these evaluations (a total of 22 
preceptors completed the included 47 evaluations). 
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Furthermore, since the entire class did not 
participate in the worl<shop, many faculty preceptors 
may have continued reviewing baseline 
expectations and skills at the beginning of the 
month and conducting rotation activities to develop 
baseline competencies instead of refining higher- 
level clinical skills. Limitations of the evaluation tool 
itself, such as the sensitivity to capture differences 
in and development of clinical skills and the inter- 
rater variability, is another potential limitation of this 
study. 

APPE performance was not determined to be 
different between the groups in this study, as all 
students appeared to achieve rotation expectations 
by the end of the rotation. This may also be 
because specific APPE activities were not designed 
based on entering levels of performance. It is 
possible that the PRW group was able to spend 
more time in patient care activities, but this pilot 
study was not designed to gather this information. 
This pilot study was unable to determine whether 
the PRW had an impact on retention of clinical skills 
at the end of the APPE year when comparing 
clinical skills between students who completed the 
PRW with the control group. This could be due to 
the relatively low sample size or it could suggest 
that despite the level of clinical skills at the start of 
the APPE year, students were achieving the 
expected level of competency by the end of the 
APPE year through their clinical rotation 
experiences. 

Overall, it does appear that providing pharmacy 
students with these additional opportunities for 
intense interaction with direct feedback from clinical 
faculty was beneficial to the participants. Each of 



the study investigators received numerous positive 
responses from the participants both at the 
completion of the PRW (less anxiety about rotations 
and faculty preceptors) and during the APPE year 
(improved patient care and APPE performance). For 
example, one student reported an instance of being 
proactive in taking a complete medication history on 
the first day of a clinical rotation. Future research 
should consider evaluating student perceptions and 
experiences in their APPE following a PRW. One 
outstanding question is to determine the optimal 
timing to introduce these and similar opportunities 
within a curriculum to further develop baseline 
professional competencies. Based on the results of 
this study, the incorporation of an introductory 
rotation is worthwhile In that It can improve students' 
baseline knowledge, skills, attitudes, and values 
required to perform the patient care responsibilities 
at the start of the APPE year. 

CONCLUSION 

The pre-rotation workshop appears to be worthwhile 
in improving student's preparation for clinical 
Advanced Pharmacy Practice Experiences, 
although it is unclear how effectively it will advance 
clinical skills overall. Additional studies will help to 
determine the role of this and other educational 
methods to help bridge didactic experiences in the 
Doctor of Pharmacy curriculum. 
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